Conventionally, a virtual reality (VR) presentation apparatus is available. The VR presentation apparatus comprises, e.g., an image display device such as a head mounted display (HMD), position and orientation detection means (e.g., a position and orientation sensor), and CG image generation means.
The position and orientation detection means is used to detect the position and orientation of the viewpoint of the observer, and to designate the position and orientation of each CG object on the virtual space. More specifically, a magnetic position and orientation sensor or the like is used. The magnetic position and orientation sensor detects the relative position and orientation between a magnetic generator (transmitter) and magnetic sensor (receiver), and FASTRAK available from Polhemus, U.S.A., or the like is known. This device detects the three-dimensional position (X, Y, Z) and orientation (Pitch, Yaw, Roll) of the sensor within a specific region in real time. When this device is attached to the HMD that the observer wears, the values of the position and orientation of the head of the observer can be detected. Also, by moving the sensor on the virtual space, the position and orientation of a CG object can be designated.
The CG image generation means lays out a CG object generated by three-dimensional (3D) modeling on the virtual space having the same scale as the physical space, and renders that virtual space based on the position and orientation of the line of sight of the observer detected by the position and orientation detection means.
By displaying a CG image generated in this way on the image display device (HMD or the like), the observer can feel as if he or she were immersed in the virtual CG space. Also, the observer can manipulate the CG object by the position and orientation sensor.
In the VR presentation apparatus, it is a common practice to complete image:update processing within a time period in which a given image frame rate or higher can be maintained, unless an image is updated in response to the behavior of the observer (a change in position and orientation of the head, manipulation of a CG object) within a given response time period.
Means for making collision and interference determination of CG objects is conventionally known, and is generally used in video games and CAD systems (for example, see Japanese Patent Laid-Open No. 6-259506).
However, the processing time required to make collision and interference determination of CG objects is influenced by the number and shapes of CG objects to be determined, the number of polygons, the number of interference points, the precision of interference determination, and the like. The determination processing can be completed within a short period of time in case of a far positional relationship, i.e., if it is determined that CG objects clearly do not interfere with each other. However, if the CG objects are close to each other or they already have an interference, many polygons which form the CG objects must be inspected in detail, resulting in a long processing time period. In some case, the processing cannot be completed within a desired frame time interval.
When the processing is not completed within the desired frame time intervals, a response to the manipulation of a CG object falls into arrears, and the observer can hardly manipulate the CG object, thus losing natural operability. For example, once the interference state has occurred and the rendering update frame rate drops extremely, the moving position of the CG object cannot be confirmed immediately if the CG object is manipulated. Therefore, the observer cannot recognize how to escape from the interference state, and can hardly manipulate the CG object. Since the manipulation of the observer is not immediately reflected in an image, natural operability required for the VR presentation apparatus is lost.
A VR system allows the user to feel the virtual space real by presenting a 3D computer graphics (CG) generated by a computer to the user. Also, in recent years, techniques for presenting information which is not available in the physical world to the user by compositing the 3D CG to the image of the physical world have been developed, and they are called an AR (Augmented Reality) system or MR (Mixed Reality) system.
The MR system can superimpose a 3D CG on a physical object. For example, in a system disclosed in Japanese Patent Laid-Open No. 2005-108108, the user can freely manipulate a virtual object by superimposing the virtual object on a physical object.
However, the system used so far is not a system which detects any interference (collision) with another virtual object upon movement of a virtual object and records an interference state. For this reason, even when the virtual objects have interfered with each other, the timing of occurrence and the accurate positions and orientations of the objects cannot be recognized.
The MR system provides a composite image obtained by compositing a physical space image and a virtual space image generated according to the viewpoint position, line of sight direction, and the like of the user. The MR system can present information to the observer as if a virtual object were existing on a physical space, and allows the observer to make an observation with a sense of actual dimensions and higher reality than a conventional virtual reality (VR) system.
In order to manipulate a CG object on the MR space, a method of associating the position and orientation of a system that measures or estimates six degrees of freedom of the position and orientation of the object on the physical space (to be referred to as a 6-degree-of-freedom sensor hereinafter) is known. For example, when a FASTRAK system available from Polhemus is used, the six degrees of freedom of the position and orientation of a receiver having a size as small as several cm2 can be measured. By associating these measurement values to an object coordinate system of the CG object, the CG object moves or rotates to follow movement or rotation of the receiver.
Using this receiver as an object manipulation device, the object manipulation device allows manipulation of the CG object.
The MR system is a technique that can present information to the observer as if a CG object were existing on a physical space. A work will be examined below wherein a small box is laid out in a large box without any lid. A physical interference between a CG object (to be referred to as CG object 1 hereinafter) of the small box which is associated with the object manipulation device and that (to be referred to as CG object 2 hereinafter) of the large box which is not associated with the object manipulation device is measured in real time. If an interference has occurred, a method of flickering the interfered portion, generating a sound, or kinesthetic sense presentation for moving the object manipulation device in a direction to avoid interference may be used.
However, the user's interestedness in this case is an interference between the interior of CG object 2 and CG object 1. In order to avoid an unwanted interference by the conventional method, CG object 1 must approach CG object 2 from its upper side.
Since both CG objects 1 and 2 are CG data and are not physical objects, no physical interference occurs on the physical space. Hence, it is desired to move CG object 1 into CG object 2 via a hole in the wall, and to then check an interference between CG objects 1 and 2. Then, the user need not perform any unwanted manipulation for approaching CG object 1 from the upper side of CG object 2.
The processing for checking an interference between CG objects requires relatively complicated processing. In some cases, as a result of the interference check processing, the display speed of an image to be presented to the user via the image observation device lowers. Consequently, it becomes difficult to perform the work itself for laying out CG object 1 in CG object 2.